Fast Tracking of Hands and Fingertips in Infrared Images for Augmented Desk Interface

In this paper, we introduce a fast and robust method for tracking positions of the centers and the ngertips of both right and left hands. Our method makes use of infrared camera images for reliable detection of user's hands, and uses template matching strategy for nding ngertips. This method is an essential part of our augmented desk interface in which a user can, with natural hand gestures, simultaneously manipulate both physical objects and electronically projected objects on a desk, e.g., a textbook and related WWW pages. Previous tracking methods which are typically based on color segmentation or background subtraction simply do not perform well in this type of application because an observed color of human skin and image backgrounds may change signi cantly due to projection of various objects onto a desk. In contrast, our proposed method was shown to be e ective even in such a challenging situation through demonstration in our augmented desk interface. This paper describes the details of our tracking method as well as typical applications in our augmented desk interface.